1. Field of the Invention
The present invention relates to an image processing apparatus for encoding image data for compression, and an image processing apparatus for decoding image data encoded for compression.
2. Description of the Related Art
As a conventional image encoding method for performing compression encoding on image data, a bidirectional predictive encoding method is commonly used in MPEG (Moving Picture Experts Group) standard. The bidirectional predictive encoding method employs three types of encoding, namely, intraframe encoding, interframe forward predictive encoding, and bidirectional predictive encoding. Corresponding to these three types of encoding, three image types, i.e. I (intra) pictures, P (predictive) pictures, and B (bidirectionally predictive) pictures, are defined. The P picture and B picture are also referred to as a non-intra picture or an inter picture.
For encoding image data to be encoded as an intra picture, the image data of interest is encoded, for example, within the same frame (or field). On the other hand, for encoding the image data to be encoded as a non-intra picture, differential image data obtained by referring to the past or future frame is encoded.
Further, hierarchical encoding, for example, is used to achieve step-wise improvement in image quality. In the hierarchical encoding method, image data to be encoded is divided into image data items in a plurality of hierarchies, and encoding is performed on the divided image data items in respective hierarchies. Note that the expression “image data items in respective hierarchies” refers to, for example, image data items divided for different frequency components.
FIG. 1 shows a configuration of an image encoding apparatus employing such a hierarchical encoding method. The image encoding apparatus includes a division circuit 170, and encoding circuits 171 and 172. The division circuit 170 is used for dividing an input image data item XX, which is an image data item to be encoded, into image data items in a plurality (two, in this example) of hierarchies (a basic hierarchy image data item X1 and a higher hierarchy image data item X2). The encoding circuit 171 encodes the basic hierarchy image data item X1 provided from the division circuit 170, while the encoding circuit 172 encodes the higher hierarchy image data item X2 provided from the division circuit 170. The encoding circuits 171 and 172 are of the same configuration, and operate in a similar manner, as described hereinafter.
The division circuit 170 includes an image processing circuit 170a and a subtracting circuit 170b. The image processing circuit 170a extracts the basic hierarchy image data item X1, which is an image data item having basic characteristics, from the input image data item XX. The subtracting circuit 170b subtracts the basic hierarchy image data item X1 from the input image data item XX. Note that the basic hierarchy image data item is an image data item that can be viewed as a normal image, and that, for example, has low frequency components in terms of spatial frequency of the image. The higher hierarchy image data item is, for example, an image data item used for obtaining a high quality image and having high frequency components. When the input image data item XX is divided with respect to spatial frequency, the image processing circuit 170a is formed by, for example, an LPF (low pass filter) circuit.
The input image data item XX is supplied to the image processing circuit 170a and the subtracting circuit 170b in the division circuit 170. The image processing circuit 170a extracts the basic hierarchy image data item X1 from the input image data item XX, and supplies the extracted basic hierarchy image data item X1 to the encoding circuit 171 and the subtracting circuit 170b. 
The subtracting circuit 170b subtracts the basic hierarchy image data item X1, which is supplied from the image processing circuit 170a, from the input image data item XX, and provides the subtraction result, i.e. differential image data, to the encoding circuit 172 as the higher hierarchy image data item X2.
FIG. 2 shows a configuration of the encoding circuits 171 and 172 shown in FIG. 1. The encoding circuits 171 and 172 are of the identical configuration and operate in a similar manner, as described above, except that different image data items to be encoded are supplied, namely, the basic hierarchy image data item X1 and the higher hierarchy image data item X2.
The encoding circuit shown in FIG. 2 includes a preprocessing circuit 151, a subtracting circuit 152, a DCT (discrete cosine transform) circuit 153, and a quantization circuit 154. The preprocessing circuit 151 rearranges image data items in, for example, a raster scan form in a predetermined order for an encoding process, and converts each of the rearranged image data items into a block scan form, thereby obtaining image data items in a plurality of macro blocks (hereinafter referred to as block data items). The subtracting circuit 152 subtracts, for example, a predictive image data described hereinafter from the image data (block data) supplied from the preprocessing circuit 151. The DCT circuit 153 converts the image data into frequency components, and performs DCT on the subtraction result supplied from the subtracting circuit 151 to acquire a DCT coefficient. The quantization circuit 154 quantizes the DCT coefficient supplied from the DCT circuit 153 based on a predetermined quantization value.
The encoding circuit further includes an inverse quantization circuit 158, an inverse DCT circuit 159, an addition circuit 160, and a frame memory 161. The inverse quantization circuit 158 inversely quantizes the data supplied from the quantization circuit 154. The inverse DCT circuit 159 performs an inverse DCT on the data supplied from the inverse quantization circuit 158 (the restored DCT coefficient). The addition circuit 160 adds the data supplied from the inverse DCT circuit 159 (image data) and predictive image data. The frame memory 161 stores the addition result of the addition circuit 160 as reference image data.
The encoding circuit further includes a motion compensation circuit 162, a motion vector detecting circuit 163, a switch 164, and a control circuit 166. The motion vector detecting circuit 163 detects a motion vector based on the image data output from the preprocessing circuit 151 and the reference image data stored in the frame memory 161. The motion compensation circuit 162 performs motion compensation on the reference image data stored in the frame memory 161 based on the motion vector detected by the motion vector detecting circuit 163 to generate reference image data for motion compensation as predictive image data.
The control circuit 166 determines whether or not to encode the image data supplied from the preprocessing circuit 151 as an intra picture, and switches the switch 164 based on the determined result.
When the control circuit 166 determines to encode the image data supplied from the preprocessing circuit 151 as an intra picture, the switch 164 is switched to supply data “0” to the subtracting circuit 152 and the addition circuit 160 based on a switching signal applied from the control circuit 166. On the other hand, when the control circuit 166 determines to encode the image data supplied from the preprocessing circuit 151 as a non-intra picture, the switch 164 is switched to supply the predictive image data generated by the motion compensation circuit 162 to the subtracting circuit 152 and the addition circuit 160 based on the switching signal applied from the control circuit 166.
Consequently, the subtraction result supplied from the subtracting circuit 152 is the image data itself when the image data is encoded as an intra picture, and is the differential image data obtained by using the predictive image data when the image data is encoded as a non-intra picture.
The encoding circuit further includes a variable length encoding circuit 155, a multiplexing circuit 156, a buffer 157, and a rate control circuit 165. The variable length encoding circuit 155 performs variable length encoding on the data supplied from the quantization circuit 154. The multiplexing circuit 156 multiplexes encoded data supplied from the variable length encoding circuit 155 (such as a DCT coefficient, a quantization value supplied from the quantization circuit 154, a picture type, and the like), the motion vector, and the like. The buffer 157 temporarily holds the data supplied from the multiplexing circuit 156, and supplies the data as a stream at a predetermined bit rate. The rate control circuit 165 monitors the data occupation state in the buffer 157, and controls the quantization value of the quantization circuit 154 in accordance with the data occupation state.
FIG. 3 is a flowchart of the encoding process performed in the variable length encoding circuit 155 shown in FIG. 2. As shown in FIG. 3, the picture type of the data (which is block data, the quantized DCT coefficient) supplied to the variable length encoding circuit 155 is determined at a step A1. When the supplied data is determined as relating to a non-intra picture, an encoding process is performed using a preset non-intra picture encoding table at a step A2 (encoding process 3).
On the other hand, when the supplied data is determined as relating to an intra picture at the step A1, the DCT coefficient type of the supplied data is determined at a step 3. This is because the DCT coefficients in respective block data items include a DC (direct current) coefficient unchanged in the block data, and an AC (alternate current) coefficient changed in the block data, and an encoding process is performed distinctively for these coefficients.
When the DCT coefficient type is determined as a DC coefficient at the step A3, an encoding process is performed using a DC coefficient encoding table preset for a differential value between the DC coefficient and that of an adjacent block data item (encoding process 1) at a step A4. On the other hand, when the DCT coefficient type is determined as an AC coefficient at the step A3, an encoding process is performed using a preset AC coefficient encoding table at a step A5 (encoding process 2).
In encoding the image data supplied to the encoding circuits 171 and 172 as an intra picture in the image encoding apparatus of the hierarchical encoding type configured as described above, motion compensation, for example, has not been performed. This is because the characteristics of the image greatly differ depending on whether the image data is encoded as a non-intra picture or an intra picture. As a result, the supplied image data itself is encoded for intra pictures, in contrast to the non-intra pictures for which differential image data having a smaller amount of data component is encoded.
In the encoding circuits 171 and 172, encoding is not performed distinctively for the input image data items (basic hierarchy image data item and higher hierarchy image data item). However, as the higher hierarchy image data is differential image data indicating the difference between the input image data and the basic hierarchy image data, the higher hierarchy image data of an intra picture closely resembles the higher hierarchy image data of a non-intra picture. Therefore, encoding the higher hierarchy image data as an intra picture without obtaining differential image data by, for example, motion compensation results in a greater encoding amount of the higher hierarchy image data, although the data component of the higher hierarchy image data is relatively smaller in amount than that of the basic hierarchy image data, leading to a decrease in encoding efficiency.